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ABSTRACT 


M 

A  common  model  for  the  changes  over  time  of  the  price  (or  sometimes  the 
logarithm  of  the  price)  of  a  commodity  Is  the  random  walk  model.  This 
is  a  Markov  model  which  supposes  that  the  change  in  price  in  any  time 
period  is  a  random  variable,  independent  of  the  past,  and  having  a  given 
distribution  F  .  In  this  note,  we  propose  a  generalized  model  in  which 
the  distribution  of  price  change  at  any  time  depends  upon  the  "environ¬ 
mental  state"  at  that  time.  That  is,  we  suppose  that  if  S  and  Y 

n  jo  t-  n 


represent  the  price  and  the  environmental  state  at  time  n  then,  given 

.T  ■  i  , .S  . .  -  S  is  a  random  variable  with  distribution  F.  .  We  also 
,vuJ^- n  n+l  n  i 

suppose  that  the  environmental  state  changes  in  a  Markovian  fashion.  An 


application  of  this  model  to  a  stock  option  example  is  presented. 


A  RANDOM  WALK  SUBJECT  TO  A  RANDOMLY  CHANGING  ENVIRONMENT 


by 

Sheldon  M.  Ross 


0.  INTRODUCTION 

A  common  model  for  the  changes  over  time  of  the  price  (or  sometimes 
the  logarithm  of  the  price)  of  a  commodity  is  the  random  walk  model.  This 
is  a  Markov  model  which  supposes  that  the  change  in  price  in  any  time 
period  is  a  random  variable,  independent  of  the  past,  and  having  a  given 
distribution  F  .  Whereas  such  a  model  might  be  appropriate  over  a  short 
time  span,  it  does  not  seem  realistic  over  a  larger  time  frame  for  it  sup¬ 
poses  that  the  future  will  behave  as  does  the  present  in  the  sense  that  the 
change  distribution  is  fixed.  More  sophisticated  models  suppose  that,  for 
each  n  ,  the  price  change  from  the  nth  to  the  (n  +  l)st  time  period 
has  its  own  distribution  Fr  ,  and  they  sometimes  even  allow  this  distribu¬ 
tion  to  depend  upon  the  price  during  the  nth  time  period.  However,  these 
models  seem  even  less  applicable  than  the  straight  random  walk  model  for 
they  assume  that  at  time  t  •  0  an  individual  would  be  knowledgeable  about 
the  price  change  distribution  at  time  n  . 

In  this  note,  we  propose  a  model  in  which  the  distribution  of  price 
change  at  any  time  depends  upon  the  "evnironmental  state"  at  that  time. 

That  is,  we  suppose  that  if  Sq  and  Yfl  represent  the  price  and  the  en¬ 
vironmental  state  at  time  n  then,  given  Y„  -  i  ,  S  . .  -  S  is  a  random 

n  n+x  n 

variable  with  distribution  F^  .  We  also  suppose  that  the  environmental 
state  changes  in  a  Markovian  fashion. 

In  Section  1,  we  present  the  specifics  of  the  above  model  and  in  Section 


2  we  compute,  conditional  on  initial  conditions,  the  mean  and  variance  of 


the  price  at  time  n  .  As,  in  most  applications,  the  "environmental  state" 
will  be  unobserved,  we  show,  in  Section  3,  how  to  derive  its  posterior  dis¬ 
tribution  at  any  time.  In  Section  4,  a  stock  option  example  is  considered. 


1.  THE  MODEL 


Let  S  denote  the  price  and  Y  the  environmental  state  at  time  n  , 
n  r  n 

where  the  set  of  possible  values  of  Yr  are  taken  to  be  the  nonnegative 
integers.  We  suppose  that  for  a  given  environmental  state  i  there  is  a 
joint  distribution  for  the  next  environmental  state  and  the  change  in  price. 
That  is,  we  suppose  there  are  transition  probabilities  and  a  family  of 

distributions  such  that 


P{Sn+l  "  Sn  -  x  ’  Yn+1  “  j  1  Yn  “  1  ’  Yn-1*  Y0  ’  Sn’  V 


PijFij(x)  * 


In  words,  if  the  present  price  is  s  and  the  environmental  state  is  i  ,  then 
the  environmental  state  will  change  to  j  with  probability  and  if 

this  occurs  the  new  price  will  be  s  plus  a  random  variable  having  distribu 
tion  .  Of  course,  it  is  not  necessary  for  us  to  think  in  terms  of  the 

environment  changing  first.  We  can  also  imagine  that,  given  Sq  ■  s  »  Yn  “ 
the  price  change  will  have  distribution  F^  defined  by 

Fi  ■  |  Vu  - 


and  given  that  the  price  change  is  x  the  next  environmental  state  will  be 
J  with  probability  P^  (x)  given  by 


Pi1dFi1(x) 
P,,(x) - iJ— - 


ij 


I  ^..(x) 


(1.1) 


y (i)  «  J  xdF^(x) 
o2(i)  =  /(x  -  y(i))2dFi(x) 

denote  the  mean  and  variance  of  the  price  change  for  environmental  state 
i  .  We  first  note  that,  in  the  long  run,  the  commodity's  price  grows  as 
a  weighted  average  of  the  p(i)  . 

Proposition  1: 

Assume  that  the  Markov  chain  with  transition  probabilities  P^  , 
i,j  >_  0  is  ergodlc  and  let  ,1^0}  denote  the  stationary  probabil 

lties.  Then,  with  probability  1, 

S 

—  "*  1  "..vd)  as  n  •*■<*>  . 
n  j  i 


Proof : 


Let  N(j,n)  denote  the  number  of  time  points  k  ,  1  £  k  <_  n  ,  for 

which  Y.  ,  •  J  .  Letting  X  ■  S  -  S  .  ,  n  >  1  ,  we  have  that 
k— l  n  n  n-l  — 


Sn  ■  S0  +  l  \ 
k-1  K 


-  so  +  I  l  \ 

i 

l<k<n 


Hence, 


n 

n 


%*l(  I 

\  l<k<n 


yiKj.n) 


Now  given  Yfe  ^  »  j  ,  has,  independent  of  the  past,  distribution  Fj 
and  so  by  the  strong  law  of  large  numbers 


I  X^/N(j,n)  u.  as  n 

k:Yk-l“J 


l<k<n 


Also,  by  the  strong  law  for  renewal  processes. 


Miisl  ^ _ * - 

n  E[time  between  visits  to  j]  j 


=  rr . 


< 


2.  MEAN  AND  VARIANCE  OF  SQ 

Let  us  suppose  that  SQ  *  s  and  Yfl  -  i  are  given  and  let  Xn  * 

S  -  S  .  ,  n  >  1  .  Hence, 
n  n-1  — 


S 

n 


I  X  +  8  . 

j-1  J 


As  j  Yj_1]  -  ,  we  see  that 


n 

E[S  ]  =  s  +  l  E[u(Y  )] 
n  j-I  J 


where  the  are  the  j  stage  transition  probabilities  of  the  Markov 

chain  [Pik]  . 

To  compute  Var  (Sn)  ,  we  use 


(2.1) 


By  the  conditional  variance  formula. 


(2.2) 


Var  (Xj)  -  e[o2(Y  )]  +  Var  [p(Y(j  -  1))] 

*  I  vll1  o2W  +  l  P^Ck))2  -  E2(X  )  . 
k  k  ^ 


Also,  we  have  for  j  <  £ 

ElXjX,]  -  I  E[XjXt  |  Yj.1  -  kjpj'1 

‘  l  f**lh  1  V x  •  Vi "  kldFx(x)pik1  • 

To  compute  E[X&  I  Xj  t  Yj-1^  ’  we  con<*ition  on  Yj  thusly: 

E‘Xt  |  X  -  x  ,  Y  -  k]  -  l  P  <x>  l  P^'W)  . 

m  r 


Hence,  for  j  <  £ 

(2.3)  >111,1  'if*  l  >W’»  I  P‘;J'1P(r)dFk(x)pJ-1 

R  <'  m  r 


where  P^Cx)  is  given  by  (1.1).  Var  (S^)  can  now  be  computed  from  (2.1), 
(2.2),  and  (2.3)  by  using 


EtXj) 


Remark: 

As  can  be  seen  by  the  above,  the  formula  for  Var  (Sn)  is  quite  involved 
when  the  number  of  environmental  states  is  not  quite  small. 


DISTRIBUTION  OF  THE  STATE  OF  NATURE 


In  most  applications,  the  state  of  nature  will  never  be  explicitly 
observed.  In  such  a  case,  we  will  suppose  that  a  prior  distribution  is  given 
for  the  initial  state  of  nature.  For  instance,  suppose 


P{Y0  =  i}  =  P°  ,  P°  >  0  ,  l  P°  =  1 


If  at  any  time  the  probability  distribution  of  the  current  state  of  nature 
is  1?  =  (P^,P2»  ...)  and  the  change  in  the  commodity's  price  is  observed  to 
equal  x  ,  then  the  next  state  of  nature  is  P(x)  =  (P^x),  ...)  where 


p.(x)  = 


l  PidF.(x)Pij(x) 
I  P±dFi(x) 


and  where  P^.(x)  is  given  by  (1.1). 
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A.  A  STOCK  OPTION  EXAMPLE 


Consider  a  stock  whose  price  changes  in  accordance  with  the  model  of 
Section  1  and  suppose  that  we  own  an  option  to  buy  one  share  of  this  stock 
at  any  time  within  N  days  for  a  fixed  price  c  .  We  need  never  exercise 
the  option  but  if  we  do  at  a  time  when  the  stock's  price  is  s  then  our 
profit  is  s  -  c  .  Under  the  assumption  that  the  environmental  state  is 
observable,  we  are  interested  in  the  strategy  that  maximizes  the  expected 
profit. 

If  we  let  V  (s,i)  denote  the  maximal  expected  profit  when  there  are 
n 

n  days  to  go  for  the  exercising  of  the  option,  the  present  price  is  s  and 
the  environmental  state  is  i  ,  then  Vn  satisfies  the  optimality  equation 


V  (s,i)  =  max 
n 


n-1 


(s  +  x, j )dF±j 


with  the  boundary  condition 


Vg(s,i)  *  max  {0,s  -  c}  . 


Lemma  2: 

(i)  V  (s,i)  -  s  is  decreasing  in  s  ,  for  fixed  i  . 

n 

(ii)  For  fixed  i  ,  V  (s,i)  is  increasing,  continuous,  and  convex  in  s 

n 

and  is  increasing  in  n  . 


Proof : 

Part  (i)  follows  by  induction.  It  is  immediate  for  n  ■  0  and  so 
assume  if  for  n  -  1  .  Now 


Vn(s,i)  -  s 


max 


[Vn_1  (s  +  x,j)  -  (s  +  x)]dFij(x)  +  y(i)(  . 


By  the  induction  hypothesis,  V  ^(s  +  x,j)  -  (s  +  x)  is,  for  each  x  , 
decreasing  in  s  and  so  (i)  follows.  Part  (ii)  also  follows  by  Induction. 
For  instance,  by  the  induction  hypothesis,  and  the  optimality  equation, 
Vn(s,i)  is  the  maximum  of  two  functions,  both  convex  in  s  ,  and  so  is 
itself  convex  in  s  . | | 

Proposition  3: 

The  optimal  policy  is  as  follows:  There  are  numbers  s(n,i)  ,  in¬ 
creasing  in  n  ,  n  >_  1  ,  such  that  the  option  should  be  exercised  when 
there  are  n  days  to  go,  the  price  is  s  ,  and  the  environmental  state  is 
i  if  and  only  if  s  ^  s(n,i)  . 

Proof : 

In  the  situation  given,  it  follows  from  the  optimality  equation  that 
it  is  optimal  to  exercise  the  option  if 

V  (s,i)  -  s  *  -c  . 
n 


Let 


s(n,i)  *  min  {s  :  V  (s,i)  -  s  ■  -c} 

n 

where  we  take  s(n,i)  to  equal  «  if  the  above  set  is  vacuous.  By  Lemma 
2,  we  have  for  all  s  >.  s(n,i) 


V  (s,i)  -  s  <_  V  (s(n,i))  -  s(n,i) 


-c 


11 


Implying  that  one  should  exercise  the  option  in  state  (s,i)  when  n  days 
remain.  That  s(n,i)  increases  in  n  follows  from  V^Cs.i)  being  in¬ 
creasing  in  n  which  is  obvious . | | 

Remark: 

It  is  easy  to  verify  that  if  u(i)  ^  0  ,  then  s(n,i)  =  “  . 

To  obtain  conditions  under  which  s(n,i)  increases  in  i  ,  we  need 
the  following  definition. 

Definition: 

The  distribution  F  is  said  to  be  more  variable  than  the  distribution 
G  if  J*f(x)dF(x)  J*f(x)dG(x)  for  all  increasing  convex  functions  f  . 

.  Theorem  4: 

Suppose  that  the  price  change  when  in  environmental  state  i  is  in¬ 
dependent  of  the  next  environmental  state.  That  is,  suppose  that  F^(t)  = 
F^Ct)  does  not  depend  on  j  .  Then  if 

(i)  F^  increases  in  variability  in  i 
and 

00 

(il)  £  P.  .  Increases  in  i  for  all  k 
j-k  1J 

then  Vn(s,i)  ,  and  thus  s(n,i)  ,  is  Increasing  in  i  . 

Proof : 

i 

! 


By  induction.  It  is  immediate  for  n  ■  0  and  so  assume  for  n  -  1  . 


12 


Now  let 


h(i,j)  *  J Vn_1(s  +  x,j)dF1(x) 


and  note  that,  for  fixed  j  ,  it  follows  from  the  monotonicity  and  convexity 
in  x  of  ^(s  +  x,j)  (from  Lemma  2)  and  condition  (i)  of  the  hypothesis 
that  h(i,j)  increases  in  i  .  Therefore, 


>  I  P,  ,h(i,j) 
j  1,3 


where  the  second  inequality  follows  from  the  induction  hypothesis,  which 
implies  that  h(i,j)  increases  in  j  ,  and  hypothesis  (ii)  which  is 
equivalent  to  the  condition  that  £  P.,k(j)  increases  in  i  whenever  k(j) 

j  13 

increases  in  j  .  Hence,  £  P  h(i,j)  increases  in  i  and  the  result 

j  3 

follows  as 


Vn(s,i)  -  max  js  -  c  ,  \  P^Mi.^l  . 


Remark: 


The  above  example  was  considered  by  Taylor  for  the  case  of  a  single 


environmental  state. 
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